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Executive Summary 

This study explored how the SAT and HSGPA can work together to help institutions to confidently admit, 
place, and support students in their academic majors while promoting student opportunity and success 
as well as institutional health and success. In particular, this study examined the differential validity of 
the SAT for predicting second-year cumulative grade point average (SYCGPA) and students’ chances of 
earning a SYCGPA of 3.0 or higher across 13 academic majors, with a special emphasis on STEM majors 
and STEM students who come from educationally disadvantaged environments. This information can be 
helpful for campus initiatives focused on major choice, academic success within major (as measured by 
grades), and retention within major. 


Correlational analyses indicated that although the relationships between the SAT and SYCGPA varied 
across the majors, the SAT was a strong predictor of SYCGPA for all 13 academic majors included in the 
study, and the SAT was stronger than HSGPA as a predictor of SYCGPA in nearly half of the academic 
majors. This finding suggests that most validity research, which does not account for students’ academic 
majors, underestimates the validity of the SAT and the incremental validity of the SAT above HSGPA. 


Logistic regression analyses found that SAT scores added information about student success at all points 
of the HSGPA scale for all 13 academic majors. For students with the same HSGPA, their chances of 
earning a SYCGPA of 3.0 or higher varied depending on their SAT scores, and this was found across all 
academic majors included in the study. Analyses also indicated that students’ chances of earning a 
SYCGPA of 3.0 or higher varied across academic majors, despite having the same SAT scores and 
HSGPAs, suggesting that college grading practices varied across academic majors and predictive models 
of student success are most useful when considered within academic departments. 


Across levels of environmental context, the inclusion of SAT scores dramatically reduced the gap in 
predicted success when compared to results found using HSGPA alone. As many institutions aim to 
diversify their STEM student bodies, they may need to be aware of academic support services that 
would be helpful to implement to ensure that all students can be successful in those majors and not just 
students from educationally advantaged environments. 


Subgroup analyses found that STEM students who came from educationally disadvantaged 
environments had lower probabilities of earning a SYGPA of 3.0 or higher even when they had the same 
SAT scores and HSGPAs of their peers, suggesting that these students may benefit from increased 
institutional support as colleges and universities seek to diversify the student body in the STEM fields. 
Paying attention to both SAT scores and environmental context information can help institutions to 
target instructional supports to the students that need them most to ensure that all students have the 
opportunity to be successful in their chosen major and have additional opportunities later on for 
graduate school and recruitment by top employers, that may in part be selected by cumulative GPA . 


Introduction 

The goal of the current study is to provide institutional SAT score users with information that can help 
them understand how the students they admit will perform in various academic majors, especially STEM 
majors, and use this information to support students appropriately. With the introduction of the new 
SAT in 2017, College Board established a validity research agenda to examine the validity of the SAT as a 
predictor of undergraduate academic performance and retention, from the beginning of college through 
graduation. As part of this ongoing series of research studies, the current study examines the validity of 
the SAT for predicting second-year cumulative grade point average (SYCGPA) across different academic 
majors, with a special emphasis on STEM majors. 


Literature Review 

Grade point average (GPA) has long stood as the collective measure of students’ academic 
performances, and it has been one of the most, if not the most, common outcome criteria of college 
education research (Beatty, Walmsley, Sackett, & Kuncel, 2015; Westrick, 2017). Though the use of GPA 
as a measure of academic performance has been ubiquitous, there has always been the 
acknowledgement— often unspoken—that undergraduate GPA represents something different for each 
student at each institution. Grading standards can vary across courses, instructors, academic semesters, 
and institutions. Though there may be some course overlap among students, what undergraduate GPA 
represents differs across students depending on the other courses they take at their institutions. In 
reality, undergraduate GPA is a unique measure for each student. 


Given the uniqueness of undergraduate GPA for each student, the use of college GPA as a common 
criterion has proven to be a challenge in predictive validity research. To address this issue, researchers 
have taken a variety of approaches to compensate for the variation in the courses taken by students and 
the grading standards across courses. Some researchers have examined validity at the individual course 
level by making grade adjustments (Berry & Sackett, 2009; Stricker, Rock, & Burton, 1993; Young, 1990a, 
1990b, 1993), and others have taken broad-based approaches that looked at grades earned by students 
in individual majors or by categorizing students by what we would now consider STEM and non-STEM 
majors (Bridgeman, Pollack, & Burton, 2008; Elliott & Strenta, 1988; Goldman & Hewitt, 1975, 1976; 
Goldman, Schmidt, Hewitt, & Fisher, 1974; Goldman & Widawski, 1976; Hewitt & Jacobs, 1978; Morgan, 
1990; Oh, 1976; Pennock-Roman, 1994; Prather & Smith, 1976; Prather, Smith, & Kodras, 1979; Shaw, 
Kobrin, Patterson, & Mattern, 2012; Strenta & Elliott, 1987). In general, these studies found that grading 
practices differed across academic majors, and that grading standards were harder in STEM majors than 
they were in non-STEM majors. 


The focus on STEM dates back at least to the Cold War and remains at the forefront of discussions on 
the current economic competition in the global economy (Institute of Medicine, 2007; National 
Academies of Sciences, Engineering, and Medicine (NASEM), 2016a; National Research Council, 2012; 
Super & Bachrach, 1957). Given the importance assigned to the identification, selection, and 
development of future scientists, engineers, and mathematicians, education researchers have focused 
on the profiles of STEM students, and the literature has shown that students in STEM majors tend to 


have higher average test scores and HSGPAs than do students in other majors (Chen, 2009; Elliott & 
Strenta, 1988; Goldman & Hewitt, 1975, 1976; Goldman, et al., 1974; Goldman & Widawski, 1976; 
NASEM, 2016b; Minaya, 2020; Nicholls, Wolfe, Besterfield-Sacre, Shuman, & Larpkiattaworn, 2007; Ost, 
2010; Pennock-Roman, 1990, 1994; Strenta & Elliott, 1987, Strenta, Elliott, Adair, Matier, & Scott, 1994, 
Super & Bachrach, 1957; White, 1992). Moreover, many of these researchers found that the key 
differentiator between STEM and non-STEM majors was mathematics performance. 


With the extended focus on STEM students, academic tilt emerged as a factor relevant to understanding 
who goes into STEM majors and careers. Academic tilt, a relative academic strength in one academic 
area as opposed to another, has been associated with students’ choices of academic majors (Coyle, 
Purcell, Snyder, & Richmond, 2014; Davison, Jew, & Davenport, 2014; Lubinski & Benbow, 2007; 
Lubinski, Webb, Morelock, & Benbow, 2001; Shea, Lubinski, & Benbow, 2001). Students with a relative 
strength in mathematics were found to be more likely to choose a STEM major in college, and students 
with a language or verbal tilt were more likely to choose a major in the humanities or social sciences. 


As described above, there has been a great deal of interest in the characteristics of students who go into 
STEM majors. In recent years, the discussions on STEM student characteristics have turned to include 
calls to increase student diversity in STEM (NASEM, 2016b, 2019). More women and underrepresented 
minorities are entering STEM fields than they did in the past, but males still outnumber females in 
engineering, mathematics, and computer science, and most of the students entering STEM fields are 
White (NASEM, 2016b). Recent College Board research has reported that underrepresented minorities 
in college are more likely to come from educationally disadvantaged environments, and students of all 
racial/ethnic groups who come from educationally disadvantaged environments perform slightly lower 
than, but about as well as, they are predicted to perform, based on their SAT scores and HSGPAs 
(Westrick, Young, Shaw, & Shmueli, 2020). Given that the literature reports that grading standards are 
stricter in STEM majors than in other majors, how STEM students from educationally disadvantaged 
environments perform in college relative to their peers is a subject that needs further investigation. 


Ultimately, examining academic predictors of college performance by academic major and how relative 
student strengths may factor into understanding performance by major, institutions can better inform 
their enrollment planning processes by academic department. This can contribute to more effectively 
fulfilling various mission-oriented initiatives to increase student diversity in particular major fields and 
more efficient planning for departmental logistics, such a course scheduling, support services to offer, 
and faculty hiring (Massa & Parker, 2007; Ohland, Sheppard, Lichtenstein, Eris, Chachra, & Layton, 
2008). In particular, this study explores how the SAT and HSGPA can work together to help institutions 
make the most informed decisions that promote student opportunity and success as well as institutional 
health and success. There will be a special focus on understanding how the use of SAT with HSGPA can 
provide the most robust information and opportunities for students from Educationally disadvantaged 
environments to enable and support their positive STEM major outcomes. 


Methodology 


Sample 


College Board broadly recruited four-year institutions with at least 250 first-year students (at least 75 of 


those students had to have SAT scores) to participate in a national SAT validity study. These institutions 


provided data through College Board’s secure online Admitted Class Evaluation Service (ACES™) system. 


Ultimately, 73 institutions provided the complete student-level information and had enough students 


within academic majors needed for the analyses that follow in this section of the report. 


Table 1 includes the characteristics of the 73 four-year colleges and universities in the sample and shows 


that the institutional sample is quite diverse with regard to region of the U.S., control (public/private), 


selectivity, and size. Compared to the population? of four-year institutions for this study, the 
institutional study sample included more public institutions, and more “large” and “very large” 


institutions than the reference population. The oversampling of larger institutions is to be expected as 


there was a sample size minimum to participate in the study. 


Table 1: Institutional Characteristics of the Study Sample and Population of Four-Year Institutions 


Study Sample 


Reference Population 
of Institutions 


Variable (k=73) (k=1,230) 
Midwest 19 (26%) 343 (28%) 
Mid-Atlantic 15 (21%) 246 (20%) 
; New England 8 (11%) 119 (10%) 
U. S. Region 
South 14 (19%) 277 (23%) 
Southwest 6 (8%) 90 (7%) 
West 11 (15%) 155 (13%) 
Public 36 (49%) 417 (34%) 
Control - 
Private 37 (51%) 813 (66%) 
AAaicanice Rate Under 25% 4 (5%) 57 (5%) 
25% to 50% 15 (21%) 211 (17%) 
51% to 75% 30 (41%) 651 (53%) 
Over 75% 24 (33%) 311 (25%) 
Small 32 (44%) 761 (62%) 
Undergraduate Enrollment Medium 10 (14%) 202 (16%) 
Large 12 (16%) 136 (11%) 
Very Large 24 (26%) 131 (11%) 


Note. k = number of institutions. Percentages may not sum to 100 due to rounding. Undergraduate enrollment was 
categorized as follows: small: 4,999 or less; medium: 5,000 to 9,999; large: 10,000 to 19,999; and very large: 20,000 or more. 


1 The population included four-year public or private nonprofit institutions that accepted 90% or fewer applicants 


for admission. 


Inclusion in the study sample required students to have redesigned SAT scores, a valid self-reported 
HSGPA, a valid first-year grade point average (FYGPA) supplied by the institution, retention to the 
second year at the same institution attended in the first year”, a second-year cumulative GPA (SYCGPA) 
supplied by the institution, and a declared major in one of the academic majors that had a sufficient 
number of students for analyses. This resulted in a sample size of 54,924 students. Table 2 provides 
more information about the characteristics of the student sample and the population of 2017 
graduating seniors who took the redesigned SAT. Compared to the population, the study sample, which 
included students who were enrolled in college and retained to the second year at the same institution, 
tended to have slightly more female students, slightly more White students and fewer Black or African 
American students and Hispanic or Latino students, and more students whose highest parental 
education level was a bachelor’s degree or higher than the overall SAT-taking population. 


Table 2: Student Characteristics of the Study Sample and 2017 Graduating Seniors with SAT Scores 


2017 Graduating 
Seniors who took 
Study Sample the SAT 
Variable (n = 54,924) (N = 1,715,481) 
24,050(84%%) | 809,462 (47%) 
Gender 
30,874 (56%) | 906,019 (53% 
American Indian or Alaska Native 129 (<1%) 7,782 (<1%) 
5,239 (10%) | 158,031.05) 
Black or African American 3,756 (7%) 225,860 (13%) 
. 9,445 (17%) | 408,067 (24% 
Race/Ethnicity - — — 
Native Hawaiian or Other Pacific Islander 66 (<1%) 4,131 (<1%) 
33,540 (61%) | 760,362 (44% 
2.119 (4%) | __57,089_ (3%) 
630 (1%) | 94,299 (5%) 
No High School Diploma 2,551 (5%) 137,437 (8%) 
High School Diploma 10,405 (19%) 482,194 (28%) 
Education Level Bachelor's Degree 21,109 (38% 473,103 (28% 
g ,109 (38%) ,103 (28%) 
Graduate Degree 16,532 (30%) 339,743 (20%) 
aoa (1%) | 108,553 (9% 


Note that previous College Board research on this 2017 cohort of students found that the overall retention rate 
to the same institution for first-year students was 83% (Westrick et. al., 2019), and that the retention rates for 
under-represented minorities were somewhat lower (Marini et al., 2019). Students not retained to the second year 
had, on average, lower SAT scores and HSGPAs compared to students who were retained to the second year. 
Consequently, aside from the correlational analyses that accounted for range restriction, the relationships 
between our predictors and other outcomes of interest, earning a SYGPA of 3.0 or higher and retention to the 
third year, are in all likelihood stronger than what was found in the current study. 


To conduct the analyses at the academic major level, students had to have a declared major at the end 
of their second year of college. Institutions reported this information for students using Classification of 
Instructional Programs (CIP) codes developed by the National Center for Education Statistics (NCES, 
2010). In the current study, we used the two-digit CIP codes and the corresponding CIP titles to 
categorize students’ academic majors. For inclusion in the study, there had to be at least 15 students 
within a two-digit CIP code at an institution. Not all academic majors are offered at every institution, so 
to avoid having small samples that may have been highly unrepresentative, we further required that an 
academic major had to be represented at 20 institutions or more for inclusion in the study. The final 
number of academic majors with at least 20 institutions with at least 15 students in a given major was 
13. These academic majors, their two-digit CIP codes, the number of institutions (k), and number of 
students (n) are presented in Table 3. 


Table 3: Academic Majors Included in the Study 


Academic Major/CIP Title (two-digit CIP code) k n 
Biological and biomedical sciences (26)* 60 8,034 
Business, management, marketing, and related support services (52) 58 10,064 
Communication, journalism, and related programs (9) 37 2,201 
Computer and information sciences and support services (11)* 34 2,403 
Education (13) 39 2,307 
Engineering (14)* 33 7,422 
English language and literature/letters (23) 26 802 
Health professions and related programs (51) 49 6,267 
Liberal arts and sciences, general studies, and humanities (24) 37 5,189 
Mathematics and statistics (27)* 20 761 
Physical sciences (40)* 34 1,518 
Psychology (42) 50 3,299 
Social sciences (45) 53 4,657 
Note. ClP=classification of instructional program; * indicates STEM majors. 


Measures 

High School GPA (HSGPA). Students’ self-reported HSGPA was obtained from the SAT Questionnaire 
when they registered for the SAT and is reported on a 12-point interval scale, ranging from 0.00 (F) to 
4.33 (At). Institutional HSGPA could not be used in this national study because it is reported on so many 
different scales across institutions. Note that the inclusion of self-reported HSGPA is consistent with 
previous admission test validity studies (e.g. Mattern and Patterson, 2014; Sawyer, 2013) and studies 
have found self-reported HSGPA to be highly correlated with actual HSGPA (Kuncel, Credé, & Thomas, 
2005; Shaw & Mattern, 2009). In the class of 2017, 93% of the SAT-taking population reported their 
HSGPA. The HSGPA measure in this study had a sample mean of 3.73 (SD=0.44). 
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SAT Scores. SAT scores were obtained from College Board’s database and matched to each student 
provided in the institution files. The SAT scores included in this study are: 


SAT Total Score (400 to 1600 scale)—increments of 10, sample mean of 1210 (SD=157). 


SAT Evidence-based Reading and Writing (ERW) Section Score (200 to 800 scale) —increments 
of 10, sample mean of 606 (SD=79). 


SAT Math Section Score (200 to 800 scale) —increments of 10, sample mean of 604 (SD=90). 


SAT Tilt. We created the SAT tilt measure to determine whether a students’ relative strength is on the 
SAT ERW section or the SAT Math section. First, we used the 2017 SAT population to create 
standardized z-scores for both ERW Section scores and Math section scores.* We then subtracted the 
SAT ERW z-score from the SAT Math z-score to get each student’s SAT tilt. Positive SAT tilt indicates that 
a student’s relative strength was on the SAT Math section. Negative SAT tilt indicates that a student’s 
relative strength was on the SAT ERW section. Most students have a small amount of SAT tilt. In the 
study sample, the mean SAT tilt was 0.00 (SD=0.63) and ranged from -3.1 to 3.6. 


Environmental Context Quintiles. Environmental context is measured by neighborhood and high school 
percentiles for six indicators related to educational opportunities and/or disadvantages students may 
experience based on where they live and learn: college attendance, crime, education level, household 
structure, housing stability, and median family income. The neighborhood level is defined by a student’s 
census tract, and the high school level is defined by the census tracts of college-bound seniors at a high 
school. Applicants from the same census tract share the same neighborhood data and indicators; 
applicants from the same high school share the same high school data and indicators. At both the 
neighborhood and high school levels, these six indicators are averaged and presented on a 1—100 scale 
to provide a neighborhood average and a high school average. A higher value on the 1—100 scale 
indicates a higher level of challenge related to educational opportunities and outcomes. For this study, 
these two averages are averaged, and then these percentiles are in turn split into quintiles, with 
students in the top 20% representing students from the most challenging environments, in Context 
Quintile 5. This is done for each quintile, with students in the bottom 20%, students from the least 
challenging environments, in Context Quintile 1.4 


College Grades. Each institution provided FYGPA and SYCGPA values for their fall 2017 first-time, first- 
year students. Across the 73 institutions in this sample, FYGPA had a sample mean of 3.21 (SD=0.59), 
and SYCGPA had a sample mean of 3.20 (SD=0.58). 


3 We used the 2017 SAT examinees who reported HSGPA (n=1,594,136) as HSGPA was required for inclusion in the 
study. 
* For detailed information on the data and methodology behind environmental context, please visit https://secure- 
media.collegeboard.org/landscape/comprehensive-data-methodology-overview.pdf. For additional information 
on Landscape, a tool that helps colleges consider environmental context in the application review process, please 
visit cb.org/landscape. 
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Descriptive Statistics 

Table 4 includes descriptive statistics for all measures of interest in the sample and for the 2017 
graduating seniors who took the SAT. As the sample includes students enrolled in college into the 
second year, it is not surprising that these students are academically stronger than the total SAT test- 
taking population across all measures. Descriptive statistics are reported for all SAT scores utilized in the 
study analyses: SAT ERW section, SAT Math section, SAT Total scores, SAT tilt, as well as HSGPA, FYGPA, 
and SYCGPA. Retention rates to the third year are also reported. ° Descriptive statistics for students 
broken out by Context Quintiles are reported in Table A1. 


Table 4: Descriptive Statistics for Measures of Interest 


2017 Graduating Seniors 
Study Sample who took the SAT 
(n = 54,924) (N = 1,715,481) 

Measure M SD Min | Max M SD | Min | Max 
SAT ERW 606 79 200 800 533 | 100 200 800 
SAT Math 604 90 200 800 527 | 107 200 800 
SAT Total 1210 157 400 |} 1600] 1060) 195 400 | 1600 
SAT Tilt 0.00) 0.63] -3.10} 3.60] 1.33] 0.68 | -5.70) 5.62 
HSGPA 3.73) 0.44] 1.33) 4.33] 3.33] 0.65] 0.00] 4.33 
FYGPA 3.21); 0.59} 0.07} 4.30 

Second-Year Cumulative GPA 3.20} 0.58] 0.05] 4.16 

Retention to Year 3 0.92} 0.27} 0.00} 1.00 

Note. Not all 2017 graduating seniors who took the SAT reported their HSGPA (n = 1,594,136). 


Table 5 contains the descriptive statistics for each of the 13 academic majors included in the study. As 
noted above, the sample consists of students who enrolled in four-year post-secondary institutions, and 
these students tend to have higher mean HSGPAs and SAT scores than students in the national 
population. Across the 13 academic majors, mean SAT ERW scores ranged 574 to 643; mean SAT Math 
scores ranged from 562 to 682; mean SAT Total scores ranged from 1140 to 1321; mean SAT tilt ranged 
from -0.47 to 0.42; mean HSGPA ranged from 3.58 to 3.89; mean FYGPA ranged from 3.07 to 3.31; 
SYCGPA ranged from 3.05 to 3.33; and third year retention rates ranged from 89% to 94%. 


STEM and Non-STEM Categories 

Interest in science, technology, engineering, and math (STEM) led to a decision to group academic 
majors into two categories: STEM and Non-STEM. Biological and biomedical sciences; computer and 
information sciences and support services; engineering; mathematics and statistics; and physical 
sciences were categorized as STEM. Business, management, marketing, and related support services; 
communication, journalism, and related programs; education; English language and literature/ letters; 


5 As noted earlier, retention to the second year was required for inclusion in the study, so the retention rate to 
year two was 100%. 
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health professions and related programs; liberal arts and sciences, general studies, and humanities; 
psychology; and social sciences were categorized as non-STEM. 


To put some of the descriptive statistics in context, we plotted the means for the thirteen academic 
majors as well as the overall sample. STEM majors are in dark blue; non-STEM majors are in light blue, 


and the overall sample is in red. In Figure 1, SAT Total score is on the X axis, and HSGPA is on the Y axis. 


STEM majors had the five highest mean SAT Total scores and five of the six highest mean HSGPAs. 
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Table 5: Descriptive Statistics by Academic Major 


Business, management, 

Academic Biological and biomedical marketing, and related Communication, journalism, Computer and information 
Major sciences support services and related programs sciences and support services Education 

(k=60, n=8,034) (k=58, n=10,064) (k=37, n=2,201) (k=34, n=2,403) (k=39, n=2,307) 
Measure Mean SD Min Max | Mean SD Min Max | Mean SD Min Max | Mean SD Min Max | Mean SD Min Max 
SAT ERW 616 77 200 800 595 76 300 800 604 75 360 800 624 79 310 800 578 76 330 790 
SAT Math 613 85 200 800 598 84 230 800 570 78 310 800 642 88 320 800 562 78 310 800 
SAT Total 1229 | 150 400 | 1590 1193 | 147 560 | 1580 1174 | 140 700 | 1570 1267 | 155 630 | 1600 1140 | 140 700 | 1540 
SAT Tilt -0.01 | 0.60 | -2.07 2.87 0.06 | 0.61 | -2.64 2.74 -0.30 | 0.60 | -2.33 2.14 0.18 | 0.62 | -1.67 3.07 -0.11 | 0.62 | -2.20 1.88 
HSGPA 3.82 | 0.40 1.33 4.33 3.66 | 0.46 1.67 4.33 3.65 | 0.46 1.67 4.33 3.73 | 0.44 1.67 4.33 3.66 | 0.46 1.33 4.33 
FYGPA 3.22 | 0.59 0.07 4.21 3.24 | 0.57 0.19 4.19 3.25 | 0.56 0.59 4.27 3.18 | 0.60 0.50 4.15 3.31 | 0.55 0.45 4.10 
SYCGPA 3.21 | 0.58 0.05 4.08 3.21 | 0.56 0.13 4.00 3.25 | 0.54 0.87 4.00 3.16 | 0.60 0.59 4.11 3.33 | 0.56 0.23 4.00 
Ret. 3 0.93 | 0.25 0.00 1.00 0.93 | 0.25 0.00 1.00 0.94 | 0.24 0.00 1.00 0.92 | 0.27 0.00 1.00 0.93 | 0.25 0.00 1.00 
Acseiic English language and Health professions and related | Liberal arts and sciences, general 
Major Engineering literature/ letters programs studies, and humanities Mathematics and statistics 

(k=33, n=7,422) (k=26, n=802) (k=49, n=6,267) (k=37, n=5,189) (k=20, n=761) 
Measure Mean SD Min Max | Mean SD Min Max | Mean SD Min Max | Mean SD Min Max | Mean SD Min Max 
SAT ERW 643 71 280 800 633 66 400 780 574 75 330 790 583 79 200 800 639 74 360 790 
SAT Math 671 74 340 800 582 72 350 790 565 81 290 800 575 86 280 800 682 75 430 800 
SAT Total 1314 | 134 630 | 1600 1215 | 124 770 | 1530 1140 | 144 690 | 1570 1159 | 153 680 | 1580 1321 | 135 850 | 1590 
SAT Tilt 0.27 | 0.56 | -1.99 3.60 -0.47 | 0.57 | -2.34 1.22 -0.04 | 0.60 | -2.38 2.59 -0.04 | 0.63 | -2.35 2.95 0.42 | 0.62 | -1.39 2.84 
HSGPA 3.89 | 0.36 2.00 4.33 3.75 | 0.42 2.00 4.33 3.71 | 0.44 1.33 4.33 3.58 | 0.48 1.33 4.33 3.88 | 0.38 2.00 4.33 
FYGPA 3.19 | 0.58 0.20 4.13 3.30 | 0.51 1.43 4.00 3.28 | 0.57 0.38 4.30 3.07 | 0.64 0.13 4.27 3.29 | 0.55 0.85 4.08 
SYCGPA 3.16 | 0.57 0.44 4.12 3.31 | 0.51 0.79 4.00 3.26 | 0.55 0.33 4.00 3.05 | 0.63 0.25 4.16 3.26 | 0.53 1.23 4.10 
Ret. 3 0.93 | 0.26 0.00 1.00 0.94 | 0.24 0.00 1.00 0.89 | 0.31 0.00 1.00 0.89 | 0.31 0.00 1.00 0.94 | 0.23 0.00 1.00 
Academic Physical sciences Psychology Social sciences 
Major (k=34, n=1,518) (k=50, n=3,299) (k=53, n=4,657) 
Measure Mean SD Min Max | Mean SD Min Max | Mean SD Min Max 
SAT ERW 632 77 330 800 592 80 370 790 619 80 330 800 
SAT Math 644 83 380 800 568 84 320 800 598 88 210 800 
SAT Total 1276 | 147 770 | 1590 1160 | 151 710 | 1540 1217 | 155 590 | 1600 
SAT Tilt 0.12 | 0.62 | -1.75 2.90 -0.19 | 0.62 | -2.34 3.15 -0.18 | 0.62 | -3.10 2.64 
HSGPA 3.83 | 0.40 2.33 4.33 3.66 | 0.46 1.67 4.33 3.70 | 0.44 1.67 4.33 
FYGPA 3.17 | 0.61 0.07 4.11 3.16 | 0.58 0.36 4.09 3.20 | 0.58 0.22 4.10 
SYCGPA 3.15 | 0.60 0.53 4.09 3.17 | 0.59 0.37 4.04 3.23 | 0.55 0.13 4.09 
Ret. 3 0.92 | 0.27 0.00 1.00 0.92 | 0.27 0.00 1.00 0.93 | 0.25 0.00 1.00 
Note. SD=standard deviation; k=number of institutions; n=number of students; ERW=Evidence-based Reading and Writing; HSGPA=high school grade point average; FYGPA=first-year grade point average; 
SYCGPA=second-year cumulative grade point average; Ret. 3= third-year retention rate. 
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Figure 1: SAT Total Score and HSGPA by Academic Major 
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In Figure 2, SAT ERW section score is on the X axis and SAT Math section score is on the Y axis. Students 
majoring in mathematics and statistics and in engineering had the highest mean SAT ERW and SAT Math 
section scores. STEM majors had the five highest mean SAT Math section scores and five of the seven 
highest mean SAT ERW section scores. 


Figure 2: SAT ERW Score and SAT Math Score by Academic Major 
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Figure 3 highlights the distribution of mean SAT tilt and mean SAT Total score across the 13 academic 
majors. As seen in Figure 1, the five STEM majors have the highest mean SAT Total scores, and we can 
also see that four of the five have mean SAT tilts toward Math. For the Non-STEM majors, seven of the 
eight have mean SAT tilts toward ERW. The greatest amount of mean Math tilt was found among 
students majoring in mathematics and statistics, and the greatest amount of ERW tilt was found among 


students majoring in English language and literature/letters. 


Figure 3: SAT Total Score and SAT Tilt by Academic Major 
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Figures 1 through 3 show that students who had declared STEM majors by the end of their second year 
of college had, on average, the highest levels of precollege academic achievement as measured by the 
SAT and HSGPA. The general trend was that the academic majors with higher mean SAT Total scores also 
had higher mean HSGPAs (Figure 1), and the academic majors with higher mean ERW section scores had 
higher mean Math section scores. However, when looking at mean SYCGPA, higher mean SAT Total 
scores and HSGPAs were not necessarily associated with higher mean SYCGPAs after controlling for 
academic major. In fact, the non-STEM majors tended to have the highest mean SYCGPAs despite having 
lower mean SAT Total scores and HSGPAs. Figures 4 and 5 show the plots for mean SAT Total scores and 
HSGPA, respectively, and SYCGPA for the 13 academic majors and the overall sample. 
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Figure 4: SAT Total Score and Second-Year Cumulative GPA by Academic Major 
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Figure 5: HSGPA and Second-Year Cumulative GPA by Academic Major 
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Figures 1 through 5 highlight the differences across academic majors regarding their students’ 
performances on precollege academic measures—the SAT and HSGPA—and the average grades these 
students earned through their first two years of college. Although there is some degree of overlap in the 
courses taken by students regardless of their academic majors, students in different programs of study 
are on different tracks and often take very different courses despite being at the same institution. As 
mentioned earlier, past research has found that grading standards vary across courses, and these 
differences manifest themselves in differences in undergraduate GPA. While it is often assumed that 
GPA is acommon criterion, in reality it is a multitude of criteria that is often unique to each student. As 
noted in the literature review, many researchers have made adjustments to student GPAs to account for 
differences in the courses students completed in college, but these adjustments are exceedingly 
complex. A more manageable solution is to conduct analyses at the academic major level, where—even 
though students still have a degree of latitude which courses they take—there are more courses in 
common within academic majors at an institution than is found when students of all academic majors at 
an institution are lumped together in overall analyses. 


Methods 

Correlational Analyses 

Correlations were computed between SAT scores (both section scores) and SYCGPA as well as HSGPA 
and SYCGPA for all academic majors. The incremental validity of the SAT over HSGPA alone was also 
evaluated for each academic major. All correlational analyses were conducted at the institution level 
and then weighted by the institutional sample size and pooled together for the entire sample. For 
correlations to be run for an academic major at an institution, there had to be at least 15 students 
within that academic major. If that was not the case, the institution was removed from that specific 
academic major analysis. Correlations were corrected for multivariate range restriction (Lawley, 1943) 
using the 2017 graduating seniors who took the SAT as the reference population. For reference, 
correlations with absolute values of .50 or higher are considered to be large, correlations with absolute 
values less than .50 and greater than or equal to .30 are considered to be medium, and correlations with 
absolute values less than .30 but greater than or equal to .10 are considered to be small (Cohen, 1988). 


After aggregating the institutional results for each academic major level, we aggregated the results from 
the 13 academic major analyses to obtain the average SAT and HSGPA correlations with SYCGPA after 
accounting for differences in course taking and grading standards across academic majors. © We also did 
this for the five STEM majors and for eight non-STEM majors. 


Finally, we ignored academic majors and calculated the average SAT and HSGPA correlations with 
SYCGPA at the institution level before aggregating across institutions. This approach is common in 
validity studies examining FYGPA. This approach did not control for differences in course taking and 


5 This approach is sometimes called a second-order meta-analysis (Schmidt & Hunter, 2015). For each academic 
major, analyses were conducted at the institution level and then aggregated. This aggregation of the results of the 
institution-level analyses is considered a first-order meta-analyses, so we had 13 first-order meta-analyses. A 
second-order meta-analysis is the aggregation of the results of the first-order meta-analyses. 
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grading standards across academic majors, and it allowed us to determine the increase (or decrease) in 
the amount of variance in SYCGPA explained by the SAT and HSGPA when considering differences across 
academic majors. 


Logistic Regression Analyses 

Logistic regression analyses were employed for predicting students’ probabilities of earning a SYCGPA of 
3.00 or higher. This criterion was selected as a reasonable threshold for indicating that students are 
performing well in their college-level work through the end of the second year of college.’ In the overall 
sample, 69% of the students earned a SYCGPA of 3.0 or higher, and a SYCGPA of 3.0 was below the 
average in every academic major in the study (see Table 5). To estimate the probability of earning a 
SYCGPA of 3.00 or higher, logistic regression analyses were conducted at each institution. The 
institution-level coefficients were weighted by the number of students in the institutional study, and 
then mean coefficients from the aggregated weights were calculated. 


Subgroup Analyses 

In addition to the traditional correlational and logistic regression approaches to examining relationships 
between precollege academic achievement measures—the SAT and HSGPA—and SYCGPA, we also 
conducted analyses that looked backward in the sense that we used SYCGPA as a starting point and 
examined the SAT and HSGPA profiles of students who were not performing as well as their peers within 
their academic majors at their institutions. Within each academic major at each institution, we used 
students’ SYCGPAs to divide them into the top, middle, and bottom third of SYCGPA. We then calculated 
the mean SAT scores and HSGPAs for these students, with an emphasis on the students in the five STEM 
majors. We next added students’ Landscape information and focused on the students within the five 
STEM majors who were in Landscape’s Context Quintiles 1 (low challenge) and 5 (high challenge). The 
purpose of these analyses was to identify students in STEM majors who may need academic support in 
their studies upon entering college.® 


Results 


Correlational Analyses 

Table 6 presents the corrected correlations for the SAT ERW section, the SAT Math section, the SAT ERW 
and SAT Math sections together, HSGPA, and both SAT sections with HSGPA all with SYCGPA. The raw 
correlations for academic majors (and other groupings) can be found in Table A2. The key findings 
follow. 


7 Some may consider a GPA of 2.5 or higher as a sufficient threshold, and this threshold has been used in previous 
College Board research that examined FYGPA and second-year retention (e.g., Westrick et al., 2019). However, in 
the current study, all students within many of the majors analyzed across multiple institutions (who were all 
retained through the end of the second year) earned a SYCGPA of 2.5 or higher, making this SYCGPA threshold 
impractical. 
8 Analyses were also conducted for students broken out by urbanicity, race/ethnicity, and highest level of parental 
education. 
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Table 6: Corrected Correlations of Predictors with Second-Year Cumulative GPA by Academic Major 


Academic Major or Grouping k n ERW | Math | SAT HSGPA | SAT+HSGPA IV 
Biological and biomedical sciences 60 8,034 .60 .62 65 .62 74 12 
Business, management, marketing, and related support services 58 | 10,064 56 56 60 62 70 08 
Communication, journalism, and related programs 37 2,201 57 54 61 .64 74 10 
Computer and information sciences and support services 34 2,403 52 55 58 59 .68 .09 
Education 39 2,307 | .51 9 55 56 67 11 
Engineering 33 7,422 .61 .67 .68 .64 17 13 
English language and literature/letters 26 802 58 56 .64 .60 74 14 
Health professions and related programs 49 6,267 55 56 .60 61 70 .09 
Liberal arts and sciences, general studies, and humanities 37 5,189 45 41 48 56 .62 .06 
Mathematics and statistics 20 761 58 57 62 .66 75 .09 
Physical sciences 34 1,518 .60 61 .66 .63 75 12 
Psychology 50 3,299 55 55 .60 55 .68 13 
Social sciences 53 4,657 58 53 .60 57 .69 12 
Weighted Average across 5 STEM Majors 60 | 20,138 59 .63 .66 .63 74 11 
Weighted Average across 8 Non-STEM Majors 72 | 34,786 54 53 58 59 .68 .09 
Weighted Average across 13 Majors 73 | 54,924 56 56 61 61 71 10 
Overall (ignoring students' academic majors) 73 | 54,924 53 51 55 59 .66 .07 


Note. k=number of institutions; n=number of students; ERW=Evidence-based Reading and Writing; HSGPA=high school grade point average; I|V=incremental 


validity. 
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First, the SAT was a stronger predictor of SYCGPA than HSGPA in six of the 13 academic majors and in 
three of the five STEM majors. After weighting and aggregating across the five STEM majors, the 
weighted average correlation between the SAT and SYCGPA was .66 versus .63 for average correlation 
between HSGPA and SYCGPA. Across the eight non-STEM majors, the mean SAT and mean HSGPA 
correlations with SYCGPA were .58 and .59, respectively. Across the 13 academic majors, both the 
weighted mean SAT and HSGPA correlations with SYCGPA were .61. In contrast, when the differences in 
courses and grading standards across academic majors were ignored at the institution level (k=73) and 
correlations were calculated for all students within institutions, the mean SAT and mean HSGPA 
correlations with SYCGPA were .55 and .59, respectively. Regardless of the approach taken, these are 
large correlations (Cohen, 1988) that show strong SAT-SYCGPA and HSGPA-SYCGPA relationships. 


Second, the SAT added incremental validity beyond that of HSGPA alone for all 13 academic majors. 
That is, the joint use of the SAT and HSGPA was better than using HSGPA alone to predict SYCGPA. The 
SAT plus HSGPA correlations with SYCGPA ranged from.62 to.77, and the incremental validity of the SAT 
beyond that of HSGPA ranged from .06 (Liberal arts and sciences, general studies, and humanities) to .14 
(English language and literature/letters). When using the weighted averages for the five STEM and eight 
non-STEM majors, the SAT plus HSGPA correlations with SYCGPA were .74 and .68, respectively, and the 
incremental validities of the SAT beyond HSGPA were .11 and .09, respectively. Across the 13 academic 
majors, the weighted average SAT plus HSGPA correlation with SYCGPA was .71, and the incremental 
validity of the SAT beyond that of HSGPA alone was .10. In contrast, when ignoring the differences in 
courses and grading standards across academic majors, the SAT plus HSGPA correlation with SYCGPA 
was .66, and the incremental validity of the SAT beyond that of HSGPA was .07. This finding suggests 
that most validity research, which does not account for students’ academic majors, underestimates the 
validity of the SAT and the incremental validity of the SAT above HSGPA. 


Logistic Regression Analyses 
Results of the logistic regression analyses conducted to predict students’ probabilities of earning a 


SYCGPA of 3.00 or higher in the 13 academic majors are presented in Figures 6 through 18. The solid 
lines represent the estimated probabilities given both HSGPA and SAT Total score. The orange dashed 
line represents the estimated probability of success when using HSGPA alone. When SAT scores and 
HSGPA are used jointly in a compensatory model, students’ chances of success vary depending on their 
SAT scores and HSGPAs. For students with the same SAT score and different HSGPAs, their probabilities 
of success will vary according to their HSGPAs. For students with the same HSGPA and different SAT 
scores, their probabilities of success will vary according to their SAT scores. However, when HSGPA is 
used alone, there is only one estimated probability of success. For example, in Figure 6 (Biological and 
biomedical sciences), when using HSGPA alone, students with a HSGPA of 4.0 have a 79% chance of 
earning a SYCGPA of 3.0 or higher. However, for students with a HSGPA of 4.0 and SAT Total scores of 
1000, 1200, and 1400, their chances of earning a SYCGPA of 3.0 or higher are 48%, 76% and 91%, 
respectively. This clearly indicates that not all HSGPAs hold the same meaning and that SAT scores 
further contextualize HSGPA for improved use on campus. Figure 19 presents students’ probabilities of 
earning a SYCGPA of 3.00 or higher based on an overall analysis that ignored students’ academic majors. 
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Figure 6: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA,; Biological and Biomedical Sciences 
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Figure 7: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Business, Management, Marketing, and Related Support Services 
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Figure 8: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Communication, Journalism, and Related Programs 
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Figure 9: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Computer and Information Sciences and Support Services 
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Figure 10: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Education 
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Figure 11: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Engineering 
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Figure 12: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; English Language and Literature/Letters 
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Figure 13: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Health Professions and Related Programs 
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Figure 14: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Liberal Arts and Sciences, General Studies, and Humanities 
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Figure 15: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Mathematics and Statistics 
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Figure 16: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Physical Sciences 
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Figure 17: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Psychology 
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Figure 18: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA; Social Sciences 
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Figure 19: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
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Table 7: Chance of Earning a SYCGPA of 3.0 or Higher given a HSGPA of 


3.7 and an SAT Total Score of 1200 


Academic Major Chance 
Communication, journalism, and related programs 90% 
Education 89% 
Health professions and related programs 84% 
Psychology 77% 
Business, management, marketing, and related support 77% 
services 

English language and literature/letters 76% 
Social sciences 75% 
Liberal arts and sciences, general studies, and humanities 69% 
Computer and information sciences and support services 66% 
Biological and biomedical sciences 65% 
Mathematics and statistics 53% 
Physical sciences 47% 
Engineering 39% 
Overall (ignoring academic major) 70% 
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As illustrated by Figures 6 through 18, students’ chances of earning a SYCGPA of 3.0 or higher varies not 
only by their SAT scores and HSGPAs, but also by their academic major. Table 7 provides students 
chances of earning a SYCGPA of 3.0 or higher given a HSGPA of 3.7 (study sample mean = 3.73) and an 
SAT Total score of 1200 (study sample mean = 1210). As shown in Table 7, students with the same 
precollege academic achievement levels have very different chances of earning a SYCGPA of 3.0 or 
higher across the 13 majors, ranging from a low of 39% (Engineering) to a high of 90% (Communication, 
journalism, and related programs). Keep in mind that these are aggregate results across institutions. 
Probabilities of success within academic majors vary across institutions, and probabilities of success vary 
across academic majors within institutions. Each institution is unique, as are the academic programs 
offered at institutions, and students’ probabilities of academic success will vary depending on the 
institution they attend and their chosen academic major, but these graphs provide some stable patterns 
and trends worth noting. 


Subgroup Analyses 

Controlling for differences across institutions by including only institutions that had all 13 academic 
majors would have excluded the vast majority of institutions. To maximize the number of institutions, 
we aggregated students into STEM and non-STEM majors. As we also wanted to examine performance 
differences across Context Quintiles, we further required that each institution had at least 15 students in 
Context Quintiles 1 (low challenge) and 5 (high challenge) in both the STEM and non-STEM groupings. 
This left us with 20 institutions for logistic regression analyses to predict students’ chances of earning a 
SYCGPA of 3.0 or higher. Descriptive statistics for these students are reported in Table A3. 


Figures 20 and 21 show the results for STEM students in Context Quintiles 1 and 5, respectively. Two 
trends stood out. First, as seen in the figures for individual academic majors, using SAT scores with 
HSGPA provides vastly more information about students’ chances of success than does using HSGPA 
alone. When using HSGPA alone, STEM students with a HSGPA of 3.7 in Context Quintile 1 (low 
challenge) had a 69% chance of earning a SYCGPA of 3.0 or higher. However, when SAT scores are 
included, the chances of earning a SYCGPA of 3.0 or higher for students with a HSGPA of 3.7 and an SAT 
Total scores of 800, 1000, 1200, 1400, and 1600 were 16%, 32%, 55%, 76%, and 89%, respectively. STEM 
students in Context Quintile 5 (high challenge) with a HSGPA of 3.7 had a 38% chance of earning a 
SYCGPA of 3.0 or higher. When SAT scores were included, students’ chances of earning a SYCGPA of 3.0 
or higher with a HSGPA of 3.7 and an SAT Total scores of 800, 1000, 1200, 1400, and 1600 were 15%, 
28%, 46%, 65%, and 81%, respectively. Despite having the same HSGPA (3.7), students’ probabilities of 
success differed dramatically depending upon their SAT scores, with probabilities of earning a SYGPA of 
3.0 or higher increasing as SAT scores increased. 
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Figure 20: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA, for STEM Students in Context Quintile 1 
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Figure 21: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA, for STEM Students in Context Quintile 5 
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Figure 22: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA, for Non-STEM Students in Context Quintile 1 
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Figure 23: Chance of Earning a SYCGPA of 3.00 or Higher, given SAT and 
HSGPA, for Non-STEM Students in Context Quintile 5 
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The second trend was that adding environmental context information tells us that students from 
educationally disadvantaged environments have lower probabilities of earning a SYCGPA of 3.0 than do 
students from educationally advantaged environments, even when they have the same HSGPAs and SAT 
scores. This indicates that precollege environmental challenges moderate the HSGPA and SAT 
relationship with SYCGPA. As noted above, when using HSGPA alone to predict students’ chances of 
earning a SYGPA of 3.0 or higher, students with a HSGPA of 3.7 in Context Quintile 1 (low challenge) had 
a 69% chance of earning a SYCGPA of 3.0 or higher, but students in Context Quintile 5 (high challenge) 
had only a 38% chance of earning a SYCGPA of 3.0 or higher despite having the same HSGPA (3.7), a 31- 
percentage point gap in predicted success. However, the inclusion of SAT scores dramatically reduces 
the gap in predicted success. For students in Context Quintiles 1 and 5 with HSGPAs of 3.7 and SAT 
scores of 800, 1000, 1200, 1400, and 1600, the gaps were 1, 4, 9, 11, and 8 percentage points, 
respectively, vastly smaller than the 31-percentage point gap found when using HSGPA alone.° The 
reason this finding is so important is that as many institutions aim to diversify the students in their STEM 
departments, they may need to be aware of academic support services that would be helpful to 
implement to ensure that all students can be successful in those majors and not just students from 
educationally advantaged environments. In other words, institutions can admit these students with 
confidence knowing that the students’ SAT scores add valuable information to their HSGPAs and the 
contextual information regarding the environments from which these students came and how they may 
be expected to perform in college. 


Figures 22 and 23 show the results for non-STEM students in Context Quintiles 1 and 5, respectively, and 
the general trends seen in these figures are the same as those found in Figures 19 and 20. Using SAT 
scores with HSGPA provides vastly more information about students’ chances of success than does using 
HSGPA alone, and environmental context information tells us that students from educationally- 
disadvantaged environments have lower probabilities of success than do students from educationally- 
advantaged environments, even when they have the same HSGPAs and SAT scores. The main difference 
between the results for the STEM and non-STEM students is that, within Context Quintiles, the non- 
STEM students tend to have higher chances of earning a SYCGPA of 3.0 or higher than do the STEM 
students given the same SAT scores and HSGPAs. 


Figure 24 shows the mean SAT total scores for the five STEM majors after breaking out students by 
SYCGPA within each academic major at the institution level. We divided them into the top, middle, and 
bottom third of SYCGPA. For example, the mean SYGPAs for the top, middle, and bottom terciles for the 
biological and biomedical sciences majors were 3.73, 3.36, and 2.55 respectively (full descriptive 
statistics for the five majors are presented in Table A4). The pattern across all five STEM majors was 
that the mean SAT Total score for the students in the middle SYCGPA group was higher than the mean 
SAT Total score for the students in the bottom SYCGPA group, and the mean SAT Total score for the 
students in the top SYCGPA group was higher than the mean SAT Total score for the students in the 


° For reference, when environmental context information was not used, STEM students (n=13,883) with a HSGPA 
of 3.7 had a 61% chance of earning a SYGPA of 3.0 or higher. When including SAT scores of 800, 1000, 1200, 1400, 
and 1600, the chances of earning a SYCGPA of 3.0 or higher were 12%, 28%, 53%, 76%, and 90%, respectively. 
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middle SYCGPA group. Across the five academic majors, the differences between the mean SAT total 
scores for students in the top third and bottom third ranged between 96 and 125 points, approximately 
half a standard deviation on the normed SAT total score scale (College Board, 2017). Analyses such as 
this, when conducted within an institution, can help institutions consider whether students with SAT 
scores in a particular range may benefit from targeted support and instruction to be more successful 
within specific majors. 


Figure 24: Mean SAT Total Score by STEM Major, Disaggregated by Top, Middle, and Bottom Thirds on 
Second-Year Cumulative GPA 
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As math coursework is a prominent characteristic of STEM majors, Figure 25 shows the mean SAT Math 
section scores for the five STEM majors after breaking out students by SYCGPA within each academic 
major at the institution level. The general pattern is the same as seen for SAT Total scores in Figure 24. 
Students with higher SAT Math section scores earned, on average, higher SYCGPAs. The differences 
between the mean SAT Math section scores for students in the top third and bottom third ranged 
between 43 and 68 points, approximately half a standard deviation on the SAT Math section score scale 
(College Board, 2017). 
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Figure 25: Mean SAT Math Score by STEM Major, Disaggregated by Top, Middle, and Bottom Thirds on 
Second-Year Cumulative GPA 
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What is noteworthy about the two figures is that the mean SAT Total scores and mean SAT Math section 
scores for students in the bottom third of SYCGPA within their academic majors at their institutions is 
that these are strong scores when looking at the national norms. Engineering students in the bottom 
third of SYCGPA had a mean SAT Total score of 1273. An SAT Total score of 1270 would place a student 
from the 2017 national cohort at the 83 SAT User percentile based on the actual scores of students in 
the graduating class of 2017 who took the new SAT (College Board, 2018). Math and statistics students 
in the bottom third of SYCGPA had a mean SAT Math section score of 656. An SAT Total score of 660 
would place a student from the 2017 national cohort at the 86" SAT User percentile (College Board, 
2018). Again, these are strong scores on the national level, but within the context of their academic 
majors, these scores are reflective of students who are not performing as well as two-thirds of their 
peers who had even higher SAT scores, on average. These trends can also be seen in SAT ERW section 
scores and HSGPA in Table A4. 


Our final set of results take the preceding analyses a step further by focusing on the STEM students in 
the bottom third of SYCGPA within their respective academic majors and subdividing them by their 
Context Quintiles. As in earlier analyses that included environmental context information, we focused 
here on students in Context Quintiles 1 (low challenge) and 5 (high challenge), and we aggregated the 
results for the five STEM majors. Descriptive statistics for the STEM students (as well as the five STEM 
majors) in Context Quintiles 1 through 5 are presented in Table AS. 
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In Figure 26, we can see that, for STEM students in the bottom third on SYCGPA, students in Context 
Quintile 1 (low challenge) have mean SAT ERW, Math, and Total scores of 630, 652, and 1282, 
respectively. For students in Context Quintile 5 (high challenge), their mean SAT ERW, Math, and Total 
scores were 547, 550, and 1097, respectively. A similar pattern can be seen in Figure 27, as the STEM 
students in Context Quintile 1 have higher mean HSGPAs, FYGPAs, and SYCGPAs than do the STEM 
students in Context Quintile 5. Notice that the HSGPA gap between the two groups was only 0.08, but 
the FYGPA gap was larger at 0.22, and the SYCGPA gap was even larger at 0.30. Also note that the mean 
SYCGPA for the STEM students in Context Quintile 5 was only 2.34, approximately an average letter 
grade of C+. Additionally, the retention rate for STEM students in Context Quintile 5 was only 82% 
compared to the 87% retention rate for system students in Context Quintile 1. From a different 
perspective, the attrition rate for STEM students in Context Quintile 5 was nearly 40% larger than the 
attrition rate for STEM students in Context Quintile 1 (18% versus 13%). This tells us that paying 
attention to both SAT scores and environmental context information can help institutions to target 
instructional supports to the students that need them most to ensure that all students have the 
opportunity to be successful in their chosen major and have additional opportunities later on for 
graduate school and recruitment by top employers, long-term outcomes that may in part be influenced 
by cumulative GPA. 


Figure 26: Mean SAT Scores by STEM Students in the Bottom Third on Second-Year Cumulative GPA 
(within major and institution), Context Quintiles 1 (Low Challenge) and 5 (High Challenge) 
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Figure 27: Mean GPAs by STEM Students in the Bottom Third on Second-Year Cumulative GPA (within 
major and institution), Context Quintiles 1 (Low Challenge) and 5 (High Challenge) 
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Discussion 

This study examined how institutions can most effectively consider and use SAT scores and HSGPA to 
understand student performance by academic major in order to maximize successful outcomes for both 
the students and the institution. The study findings offer insight about differences in departmental 
grading practices and how those differences need to be factored into predictive models, and how the 
use of SAT, HSGPA, and student’s environmental context information can jointly inform advising 
decisions as well as the implementation of targeted academic support to promote student opportunities 
and success in different majors. 


The descriptive statistics confirm trends that have existed for decades: STEM majors tend to have higher 
levels of precollege academic achievement, but lower college grades compared to their non-STEM 
peers. These differences suggested differential grading standards across academic majors, which 
previous research has also found, and the logistic regression analyses indicated that students with the 
same SAT scores and HSGPAs had different chances of earning a SYCGPA of 3.0 or higher across the 13 
academic majors included in the study. 


Regardless of differences in grading standards, the correlational analyses found that the SAT was a valid 
predictor of SYCGPA for all 13 academic majors included in the study, and the SAT always added 
incremental validity beyond that of HSGPA. Moreover, the SAT was stronger than HSGPA as a predictor 
of SYCGPA in nearly half of the academic majors, including three of the five STEM majors. This is in 
contrast to overall analyses that ignore academic majors and show HSGPA to be slightly stronger than 
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the SAT as a predictor of undergraduate GPA. However, the results of overall and academic major 
analyses do agree in an important way: Overall and for all 13 academic majors, the joint use of the SAT 
and HSGPA was always better than using the SAT or HSGPA alone to predict SYCGPA. Accurately 
understanding how students may be expected to perform in a major not only helps an institution to 
appropriately plan for departmental resources but to most effectively target academic support and 
additional instruction to those students most likely to benefit from it. 


Another key finding from the correlational analyses was that conducting analyses at the academic major 
level increased the average validity of both SAT scores and HSGPA when compared to the mean 
validities of these predictors when institution level correlational analyses ignored students’ academic 
majors. Each student’s GPA is a unique measure, but in practice GPA is usually considered a common 
metric despite students taking different courses with different grading standards. A more-refined study 
would examine the validity of the SAT and HSGPA as predictors of individual courses within an 
institution, followed by weighting and aggregating the results at the institution level, and followed by 
weighting and aggregating results across institutions. An alternative would be to make grade 
adjustments based on course difficulty estimates at the institution level, calculate adjusted GPAs for 
each student at the institution level, calculate correlations at the institution level, and then weight, 
aggregate, and average the correlations across institutions. These alternative approaches are 
exceedingly complex, and conducting analyses at the academic major level proved sufficient to show 
that the predictive strength of the SAT and HSGPA are stronger than many believe. 


The final set of analyses add to our knowledge of students from educationally disadvantaged 
environments that have declared a STEM major. Previous research found that students from 
educationally disadvantaged environments, even when they have the same SAT scores and HSGPAs as 
their peers, earn grades slightly lower than predicted using their SAT scores and HSGPAs (Westrick et al., 
2020). The supplemental analyses in the current study confirmed the finding that precollege 
environmental challenges moderate students’ academic performances in college. This added SAT 
information can help institutions differentiate which of their admitted students will be most likely to 
benefit from extra support to ensure that these students persist in their studies, graduate, pursue post- 
graduate opportunities in STEM (if desired), and contribute to the STEM fields in the coming decades. 


Conclusion 

To summarize, findings from the current study affirm the value and effectiveness of the SAT as a tool for 
institutions to use to inform admission, academic advising, and instructional support decisions by 
academic major. Results show that: 


e SAT scores are strongly predictive of college performance across multiple academic majors— 
students with higher SAT scores are more likely to have higher grades in college. 

e The SAT was stronger than HSGPA as a predictor of SYCGPA in nearly half of the academic 
majors. 
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e = The joint use of the SAT and HSGPA was always better than using the SAT or HSGPA alone to 
predict SYCGPA. 

e For students with the same HSGPA, their chances of earning a SYCGPA of 3.0 or higher varied 
depending on their SAT scores. This was found across all academic majors included in the study. 

e Students’ chances of earning a SYCGPA of 3.0 or higher varied across academic majors, 
suggesting that grading practices varied across academic majors. 

e The SAT can be used to increase the diversity of the student body in STEM fields by helping 
institutions to confidently admit the students they are most interested in to shape a class in 
keeping with their mission and goals, while remaining fully aware of which students may benefit 
from additional instructional support in the major field in order to maximize their success in 
college and beyond. 


In addition to studying these SAT and HSGPA relationships from a national perspective as part of a 
comprehensive SAT validity research agenda, College Board also provides a free online service for higher 
education institutions and systems (Admitted Class Evaluation Service, ACES) to conduct campus or 
system-specific validity studies (with outcomes such as FYGPA, course grades, retention, and 
completion) that meet their specific institutional needs and help to provide greater context and 
meaning for the use of College Board scores in different enrollment management processes. 
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Appendix 


Table A1: Descriptive Statistics by Context Quintiles 


Context Quintile Measure n Mean SD Min Max 
SAT ERW 20,774 628 73 330 800 

SAT Math 20,774 633 84 200 800 

SAT Total 20,774 1261 144 550 1600 

Context Quintile 1 SAT Tilt 20,774 0.06 | 0.61 -2.45 3.60 
HSGPA 20,774 3.73 | 0.42 1.33 4.33 

FYGPA 20,774 3.30 | 0.55 0.07 4.27 

SYCGPA 20,774 3.29 | 0.53 0.13 4.16 

Year 3 Retention 20,774 0.94 | 0.23 0.00 1.00 

SAT ERW 12,575 614 74 200 800 

SAT Math 12,575 610 84 280 800 

SAT Total 12,575 1223 145 680 1600 

intil SAT Tilt 12,575 -0.02 | 0.63 -2.38 3.07 
PantexPeuintle: | Asana 12,575| 3.75| 043| 167| 4.33 
FYGPA 12,575 3.24 | 0.56 0.57 4.30 

SYCGPA 12,575 3.24 | 0.54 0.23 4.11 

Year 3 Retention 12,575 0.93 | 0.25 0.00 1.00 

SAT ERW 9,811 600 76 200 800 

SAT Math 9,811 594 85 200 800 

SAT Total 9,811 1195 148 400 1590 

Context Quintile 3 SAT Tilt 9,811 -0.03 | 0.63 -2.64 2.64 
HSGPA 9,811 3.76 | 0.44 1.33 4.33 

FYGPA 9,811 3.19 | 0.59 0.07 4.27 

SYCGPA 9,811 3.17 | 0.59 0.05 4.10 

Year 3 Retention 9,811 0.91 | 0.28 0.00 1.00 

SAT ERW 6,826 582 79 310 800 

SAT Math 6,826 572 85 210 800 

SAT Total 6,826 1154 151 580 1590 

intil SAT Tilt 6,826 -0.06 | 0.64 -3.10 2.90 

PONENT UIN  aSGpA 6826|  3.72| 047| 167] 4.33 
FYGPA 6,826 3.10 | 0.62 0.13 4.30 

SYCGPA 6,826 3.08 | 0.62 0.13 4.14 

Year 3 Retention 6,826 0.90 | 0.30 0.00 1.00 

SAT ERW 4,938 539 78 280 790 

SAT Math 4,938 529 81 250 800 

SAT Total 4,938 1068 146 560 1570 

Context Quintile 5 SAT Tilt 4,938 -0.03 | 0.63 -2.35 2.81 
HSGPA 4,938 3.61 | 0.50 1.33 4.33 

FYGPA 4,938 2.95 | 0.64 0.19 4.08 

SYCGPA 4,938 2.92 | 0.65 0.19 4.00 

Year 3 Retention 4,938 0.87 | 0.33 0.00 1.00 


Note: n=number of students; SD=standard deviation; ERW=Evidence-based Reading and Writing; 
HSGPA=high school grade point average; FYGPA=first-year grade point average; SYCGPA=second-year 
cumulative grade point average. 
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Table A2: Raw Correlations of Predictors with Second-Year Cumulative GPA by Academic Majors 


Academic Major or Grouping k n ERW Math SAT HSGPA | SAT+HSGPA IV 
Biological and biomedical sciences 60 8,034 .36 39 44 36 52 16 
Business, management, marketing, and related support services 58 10,064 30 30 35 37 AT 10 
Communication, journalism, and related programs 37 2,201 34 .29 39 43 54 11 
Computer and information sciences and support services 34 2,403 .27 31 35 35 47 12 
Education 39 2,307 32 .28 37 38 49 11 
Engineering 33 7,422 .29 38 40 32 48 16 
English language and literature/letters 26 802 33 31 42 38 54 16 
Health professions and related programs 49 6,267 33 33 38 38 49 11 
Liberal arts and sciences, general studies, and humanities 37 5,189 .26 21 .29 38 44 .06 
Mathematics and statistics 20 761 32 .27 .40 .40 52 12 
Physical sciences 34 1,518 .37 38 44 37 54 17 
Psychology 50 3,299 33 33 .40 33 48 15 
Social sciences 53 4,657 37 .28 39 33 48 15 
Weighted Average across 5 STEM Majors 60 20,138 .32 .37 Al 35 50 15 
Weighted Average across 8 Non-STEM Majors 72 34,786 .32 .29 36 37 48 11 
Weighted Average across 13 Majors 73 54,924 .32 .32 38 36 49 13 
Overall (ignoring students' academic majors) 73 54,924 31 .29 34 36 44 .08 
Note: k=number of institutional studies; n=number of students; ERW=Evidence-based Reading and Writing; HSGPA=high school grade point average; IV=incremental validity. 
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Table A3: Descriptive Statistics for Students Categorized as STEM or Non-STEM, Overall and by Context Quintiles 


Context SAT ERW SAT Math SAT Total SAT Tilt HSGPA FYGPA SYCGPA Ret. 3 
Group Quintile n Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean 
cQi 4,636 | 653 62 675 70 1328 118 | -0.21 | 0.59 | 3.85 | 0.36 | 3.31 | 0.55 | 3.28 | 0.54 | 94% 
CQ2 3,303 | 638 66 653 76 1291 128 | -0.14 | 0.61 | 3.87 | 0.37 | 3.23 | 0.56 | 3.20 | 0.55 | 94% 
ih CQ3 2,788 | 626 70 639 78 1264 133 | -0.13 | 0.62 | 3.86 | 0.37 | 3.18 | 0.58 | 3.14 | 0.58 | 92% 
CQ4 1,905 | 608 75 614 83 1221 144 | -0.08 | 0.63 | 3.84 | 0.41 | 3.07 | 0.62 | 3.04 | 0.62 | 91% 
CQ5 1,251 | 559 78 564 81 1123 146 | -0.10 | 0.63 | 3.71 | 0.46 | 2.88 | 0.62 | 2.84 | 0.65 | 89% 
Overall 13,883 | 629 73 644 83 1273 143 | -0.15 | 0.61 | 3.84 | 0.38 | 3.19 | 0.59 | 3.16 | 0.59 | 93% 
cQi 7,197 | 617 67 614 76 1232 128 0.01 | 0.63 | 3.67 | 0.42 | 3.31 | 0.54 3.3 0.52 | 94% 
CQ2 5,158 | 610 68 596 75 1207 127 0.11 | 0.63 | 3.72 | 0.41 | 3.26 | 0.56 | 3.26 | 0.53 | 94% 
Non- CQ3 4,313 | 596 71 580 74 1176 129 0.12 | 0.63 | 3.72 | 0.43 | 3.21 | 0.58 | 3.19 | 0.57 | 93% 
STEM ca4 3,086 | 578 75 559 77 1137 138 0.14 | 0.63 | 3.71 | 0.46 | 3.14 | 0.59 | 3.11 | 0.59 | 91% 
CQ5 2,209 | 534 77 517 74 1052 138 0.09 | 0.62 | 3.59 | 0.49 | 2.98 | 0.62 | 2.93 | 0.63 | 89% 
Overall 21,963 | 598 75 586 81 1183 142 0.08 | 0.63 | 3.69 | 0.43 | 3.22 | 0.57 | 3.21 | 0.57 | 93% 


Note: n=number of students; SD=standard deviation; SYCGPA=second-year cumulative grade point average; ERW=Evidence-based Reading and Writing; HSGPA=high school 
grade point average; Ret.3=third-year retention rate. 
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Table A4: Descriptive Statistics for Students Disaggregated by Second-Year Cumulative GPA Thirds within Academic Majors 


SYCGPA ERW Math Total Score SAT Tilt HSGPA Ret. 3 

STEM Major Third n mean SD mean SD mean SD mean SD mean SD mean SD mean 
a Top 1,723 | 3.73 | 0.21 | 646 | 68 | 651 | 75 | 1298 | 130 | 0.05 | 057 | 3.97 | 0.32 | 97% 
ae Middle | 1,738] 3.26 | 0.26 | 614 | 69 | 607 | 76 | 1221 | 131 | -0.04 | 0.62 | 3.83 | 0.36 | 97% 
Bottom | 1,724] 2.55 | 0.50 | 591 | 71 | 585 | 80 | 1176 | 139 | -0.03 | 0.60 | 3.67 | 0.42 | 86% 

Coniputerana Top 604| 3.71 | 0.19 | 643 | 71 | 667 | 82 | 1310] 139 | 0.23 | 063 | 3.89 | 0.33 | 97% 
information sciences _| Middle 618 | 3.23 | 0.23 | 621 | 73 | 634 | 82 | 1255 | 140 | 0.14 | o64 | 3.72 | 0.42 | 97% 
and support services | Bottom | 607| 2.49 | 0.47 | 601 | 74 | 615 | 80 | 1216 | 141 | 0.16 | 062 | 3.57 | 0.45 | 84% 
es Top 1,740 | 3.73 | 019 | 667 | 66 | 701 | 66 | 1368 | 119 | 0.31 | 0.55 | 4.01 | 0.29 | 95% 
BNEIIEE HME Middle | 1,748| 3.20 | 0.21 | 640 | 64 | 669 | 67 | 1308 | 118 | 0.27 | 057 | 3.90 | 0.34 | 95% 
Bottom | 1,743] 2.52 | 0.42 | 625 | 70 | 648 | 72 | 1273 | 129 | 0.23 | 058 | 3.79 | 0.38 | 87% 

| Top 187 | 3.72 | 019 | 659 | 73 | 701 | 77 | 1360 | 133 | 0.39 | 0.66 | 4.01 | 0.30 | 95% 
ong and Middle 193 | 3.26 | 0.24 | 624 | 75 | 670 | 67 | 1293 | 126 | 0.45 | 0.64 | 3.87 | 0.36 | 96% 
Bottom 192| 2.65 | 044 | 608 | 70 | 656 | 79 | 1264 | 135 | 0.47 | 0.64 | 3.69 | 0.44 | 89% 

Top 352 | 3.71 | 0.22 | 661 | 73 | 679 | 77 | 1340 | 134 | 0.16 | o64 | 3.94 | 0.34 | 97% 

Physical sciences Middle 359 | 3.16 | 0.23 | 619 | 73 | 629 | 80 | 1248 | 139 | 0.11 | o62 | 3.85 | 0.35 | 96% 
Bottom | 355| 2.45 | 0.45 | 603 | 72 | 613 | 76 | 1216 | 134 | 0.12 | 0.64 | 3.67 | 0.41 | 83% 


Note: n=number of students; SD=standard deviation; SYCGPA=second-year cumulative grade point average; ERW=Evidence-based Reading and Writing; HSGPA=high school 
grade point average; Ret.3=third-year retention rate. 


42 


Table A5: Means (SDs) for Measures for STEM Students in the Bottom Third of SYCGPA within Academic Majors, by Context Quintiles 


Context SYCGPA SAT ERW SAT Math SAT Total SAT Tilt HSGPA FYGPA Ret 3 

STEM Major Quintile n Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean 
1 308 | 265 | O46 .| 617 52 617 74 | 1234 | 123 | 0.01 | 059 | 3.64 | 042 | 2.72 | 0.48 | 86% 

Biological and 2 378 | 2.61 | 0.46 | 604 67 598 76 | 1202 | 129 | -003 | 060 | 3.70 | 043 | 2.71 | 0.51 | 88% 
biomedical 3 362 | 2.59 | 0.50 | 596 65 591 69 | 1187 | 120 | -0.02 | 060 | 3.72 | 0.38 | 2.67 | 0.54 | 85% 
sciences a 337 | 247 | 053 | 581 69 567 73 | 1148 | 133 | -010 | 060 | 3.69 | 0.43 | 2.58 | 0.52 | 85% 
5 249 | 2.34 | 051 | 538 72 528 30 | 1066 | 139 | -0.02 | 0.60 | 359 | 0.45 | 2.49 | 0.50 | 87% 

Computer and 1 183 | 2.53 | 046 | 623 69 647 67 | 1270 | 123 | 0.25 | 0.56 | 3.55 | 0.40 | 2.62 | 0.55 | 83% 
information I 124 | 2.59 | 046 | 606 64 610 74 | 1216 | 124 | 0.07 | 058 | 3.60 | 0.47 | 2.64 | 0.49 | 87% 
sciences and 3 144 | 2.44 | 047 | 605 70 618 85 | 1223 | 139 | 015 | 068 | 358 | 0.46 | 2.59 | 0.49 | 86% 
support a 93 | 2.39 | 0.51 | 586 75 590 76 | 1176 | 135 | 0.07 | 067 | 3.54 | 048 | 2.49 | 0.53 | 76% 
services 5 63 | 2.46 | 041 | 541 81 Gs 83 | 1104 | 152 | 0.27 | 0.62 | 3.60 | 0.53 | 2.62 | 0.52 | 86% 
1 537] 259 | 0.38 | 644 57 675 57 | 1319 | 98 | 0.30 | 056 | 3.77 | 0.34 | 2.72 | 0.45 | 88% 

2 407 | 2.57 | 0.39 | 639 61 662 61 | 1301 | 106 | 0.22 | 059 | 3.83 | 0.37 | 2.67 | 0.44 | 90% 

Engineering 3 354 | 2.54 | 038 | 627 68 648 64 | 1275 | 118 | 0.20 | 058 | 3.85 | 0.35 | 2.64 | 0.41 | 87% 
4 260 | 2.51 | 0.46 | 610 74 622 78 | 1232 | 140 | 013 | 058 | 3.80 | 0.41 | 2.62 | 0.50 | 87% 

5 185 | 2.23 | 048 | 558 78 573 77. | 1132 | 142 | 019 | 060 | 3.65 | 0.46 | 2.42 | 0.49 | 76% 

1 ba | 279 | 035 | 696 60 699 54 | 1325 | 93 | 0.70 | 064 | 3.73 | 0.31 | 2.79 | 0.40 | 96% 

eee 2 37 | 267 | 0.48.| 636 58 666 74 | 1302 | 120 | 0.29 | 055 | 3.68 | 0.38 | 2.76 | 0.57 | 89% 
pera 3 46| 271 | 037 | 612 61 658 65 | 1270 | 103 | 0.46 | 0.71 | 3.83 | 0.34 | 2.83 | 0.48 | 85% 
4 28 | 2.63 | 043 | 605 70 647 74 | 1252 | 134 | 042 | 052 | 3.71 | 063 | 2.69 | 0.58 | 96% 

5 27 | 2.29 | 047 | 531 70 563 81 | 1094 | 135 | 0.37 | 067 | 3.36 | 0.51 | 2.48 | 0.37 | 74% 

1 101 | 2.49 | 042 | 624 69 649 72 | 1273 | 129 | 0.26 | 055 | 3.57 | 0.43 | 2.58 | 0.52 | 89% 

ee 2 75 | 2.51 | 041 | 619 74 631 67 | 1250 | 122 | 013 | 0.70 | 3.72 | 0.37 | 2.62 | 0.49 | 85% 
meee 3 70 | 2.45 | 049 | 598 63 609 60 | 1207 | 107 | 0.13 | 0.60 | 3.74 | 0.39 | 2.60 | 0.46 | 80% 
4 59 | 2.42 | 045 | 5384 66 585 73. | 1169 | 119 | 0.06 | 071 | 3.73 | 0.38 | 2.55 | 0.54 | 78% 

5 50 | 2.29 | 046 | 568 78 550 72 | 1119 | 137 | -012 | 061 | 3.64 | 0.47 | 2.47 | 0.44 | 80% 

1 1,273 | 2.60 | 042 | 630 62 652 70 | 1282 | 118 | 0.22 | 060 | 3.68 | 0.39 | 2.70 | 0.48 | 87% 

a 1,021 | 2.59 | 043 | 620 66 630 76 | 1250 | 127 | 0411 | 061 | 3.74 | 0.41 | 2.68 | 0.48 | 89% 

STEM Overall 3 976 | 2.54 | 0.45 | 610 68 620 74 | 1230 | 127 | 012 | 062 | 3.76 | 039 | 2.65 | 0.48 | 85% 
4 777 | 2.48 | 050 | 593 72 592 82. | 1185 | 140 | 0.03 | 062 | 3.71 | 0.44 | 2.58 | 0.52 | 85% 

5 B74 | 231 | 049 | 5a7 76 550 81 | 1097 | 144 | 0.09 | 062 | 3.60 | 047 | 2.48 | 0.49 | 82% 


Note: SYCGPA=second-year cumulative grade point average; ERW=Evidence-based Reading and Writing; HSGPA=high school grade point average; FYGPA=first-year grade 


point average; n=number of students; SD=standard deviation; Ret.3=third-year retention rate. 
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